Apparatus, method and system for microsurgical suture training

ABSTRACT

Systems and methods of microsuture training include receiving an image or video of a suture; extracting features of the suture from the image or video; scoring the suture based on the extracted features of the suture and based on a rating and complexity of the suture to determine a quality of the suture in an objective manner. A suture training system includes a suture training apparatus configurable with a plurality of orientations of simulated or actual tissue to simulate natural anatomical orientations encountered in actual surgery, wherein the plurality of simulated tissue orientations are objective based on fiducial markings on the suture training apparatus; and an image analysis system configured to receive an image or video of a suture performed on the suture training apparatus; extract features of the suture from the image or video; score the suture.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application/patent is a divisional of U.S. patentapplication Ser. No. 13/667,066, filed Nov. 2, 2012, and entitled“Apparatus, Method and System for Microsurgical Suture Training,” thecontents of which are incorporated by reference herein.

FIELD OF THE DISCLOSURE

The invention relates to an apparatus for microsurgical suture training,a method of processing a microsuture training image, and a microsuturetraining.

BACKGROUND OF THE DISCLOSURE

There is a set of basic surgical moves, which, when sequenced forms asurgical procedure. A fundamental difference between surgical proceduresis the orientation of the surgical target upon which basic surgicalmoves are performed. For example, anastomosis is a key surgicalprocedure used in establishing continuity in the blood vessels. Itconsists of a series of steps to suture a vessel or a nerve, startingfrom grasping the tool, entering the tissue, exiting and tying the knot.However, depending on the orientation of the vessel, the basic movementsbecome more complex. The practice of the surgical tasks such as toolgrasping and suture placement is important so is practicing them indifferent anatomical orientations encountered in surgery. The practiceof these surgical moves in natural anatomical positions improves surgerytraining.

Medical, dental and veterinary students, as well as more experienceddoctors, dentists and veterinarians, learning new surgical techniques,must undergo extensive training before they are optimally qualified toperform surgery. It would be advantageous for students and medicalpersonnel to obtain as much hands-on experience as possible, operatingon actual or simulated body structures when learning surgicalprocedures. These skills are generally learned by observation anddidactic instruction from an accomplished surgeon tutor. Learning ofthese basic skills can be enhanced by viewing video presentations ofprocedure-specific instructions. Such practice can shorten the learningcurve in the operating room.

For example, a student may perform procedures on human cadavers oranimals. Both are expensive. In addition, unnecessary surgery on animalsis resisted for ethical and legal reasons. Moreover, objectiveassessment of surgical skill is difficult on animals or cadaver. Asurgeon may be trained to a new surgical procedure, but the amount oftraining time is very insufficient to perfect the practice. Moreover,the training needs to be continuous to sustain the skill. Therefore,even practicing surgeons need a tool to train them outside the operatingroom. It is observed that an experienced surgeon makes, the less amountof and less number of movements to complete a surgical procedure,resulting less trauma of the surrounding tissue and improved healingtime.

Presently, there exists virtual and non-virtual simulators on which topractice surgical skills. Most virtual simulators rely on sophisticatedhaptic sensors and software integrated with large computer systems thatare immobile and often extremely expensive. Teaching institutions thatcan afford them are usually only able to purchase a limited quantity.Therefore, students often have restricted access and limited times topractice surgical techniques using virtual simulators. In addition,virtual simulators are used for more specialized and complex surgerytechniques. For example, endoscopic vessel harvesting systems which aremade to model a specific procedure are available. In these specializedprior-art systems, a specific procedure, for example, harvesting of thesaphenous vein is modelled. Hence it cannot be adapted to practice newersurgery techniques since it does not teach the basic surgical skills.Basic surgical skills include cutting, knot-tying techniques, suturingtechniques, dissection, clamping, clipping, grasping, ligating,cannulation, stapling, cauterization, and suture cutting, among others.

Repetitive practice of these skills is necessary to achieve competencyand subsequent mastery characterized by rapidity, automaticity, anddelicacy. Coordinated motions of both hands to move and stabilizetissues with the non-dominant hand and precise cutting, clamping, orsuturing by the dominant hand are characteristic of most basic surgicaltasks. A surgical simulation system ideally should provide facility topractice these basic skills.

Though there are some training tools and kits available, theeffectiveness of such training tools are limited due to the disparitybetween the actual surgery and the surgery training. For example,Lumely's practice block allows synthetic vessels to be placed, dividedand sutured for practice. The practice is limited to a 2D orientationand hence is of limited use. In real life surgery, the anatomicalorientations are more complex and there is no method for objectiveanalysis of the quality of the suturing.

Many other existing practice systems focus on providing real lifesimulations such as blood flow, but do not focus on improving surgeon'sskills to manipulate the tools under anatomical orientations thatrestrict surgeon's movement. Many of the available systems present thesimulated tissue in the horizontal plane, such as natural or syntheticspecimens prepared and presented in a convenient horizontal plane. Forexample, latex sheets provided by ‘Braun’ allow simulation of suturingin various angles, but are limited to the horizontal plane.

A similar device called the anastomosis simulator is marketed bySharpoint, which uses silicone tubes that simulate vessel anastomosis.It is also limited to simulation in one horizontal plane.

Existing systems ignore the fact that coordinated hand movements aredifficult when surgical tissues are oriented in oblique or verticalplane, which is a more natural presentation of the clinical task. Hence,existing surgery training systems do not pose similar type of dexteritychallenges as in a real surgery.

BRIEF SUMMARY OF THE DISCLOSURE

Existing surgery training systems are limited to training the surgeonsin a convenient horizontal plane. They do not pose dexterity challengesof a real surgery where the tools have to be operated on vesselsoriented in oblique or vertical plane. Hence a microsurgery trainingtool that offers similar dexterity challenges of actual surgery may bedesirable for sharpening surgical skills.

Given the finer nature of the suture in microsurgery, identifying thegood suture from bad suture can be difficult and time consuming. Theidentification of good suture from bad suture is important to depict theskill profile over time of practice. There are a number of types oferrors that a trained surgeon may make. Identifying them may assist infixing the errors and advising the right technique.

It is challenging in grading the surgery skills in-relation to a groupof surgeons practicing in the same or similar facilities. Given enoughtraining, each surgeon of the group must practice at the same level.Ranking can be used by surgeons to achieve better surgical skill incomparison to their peers. For a comparison to be possible, all surgeonsin the population must be facing standardized dexterity challenges.

Given the finer nature of the suture in microsurgery, identifying a goodsuture from bad suture can be difficult and time consuming. Theevaluation of the quality of suturing is manual and subjective. Thesuccess of the surgery depends on the suturing skills of the surgeon inclosing the wound to prevent infection and creating appropriate strengthon the tissues to enhance healing. A good suture leaves less scarring,and good sutures produce eversion of the tissue. There is a need forobjective and standardized evaluation of the quality of the suture.

The silicone tubes used in existing systems do not allow for easyassessment of individual stitch or the spacing between the sutures. Bothstitch and stitch spacing are parameters of assessing the accuracy ofstitch placement and expertise in suturing.

In general terms in one aspect the present invention proposes a suturetraining apparatus configurable with a plurality of simulated tissueorientations to simulate natural anatomical orientations encountered inthe actual surgery. This may be provided with two joint connections toallow 6 degrees of freedom. The apparatus allows manual orientation ofthe simulated tissue allowing the surgeon to acquire necessary skillsfor placing sutures in various planes of orientation.

It utilizes the observation that the human vascular system presentsvarious three dimensional orientations for the vessels and consequentlya number of three dimensional orientations for the surgeons to adapt thesuturing technique to operate on the vessels. For example, end-to-endanastomosis requires orienting the tools orthogonal to the direction ofthe vessels, whereas end-to-side anastomosis requires orienting thetools in an oblique orientation to the vessels. By using the apparatus,the surgeon can orient the vessel specimen in any natural anatomicalorientation to have a similar tool and hand orientation as in the caseof a real surgery.

It is capable of creating complex anatomical orientations encountered inanastomosis process. A flexible synthetic tissue strip is mounted onclips and oriented variously through an articulated arm. A series ofpreset orientations are calibrated to simulate various clinical patientpostures and surgery sites. The cut edge of the synthetic tissue stripsimulates a segment of the circumference of the vessel and synthetictissue strip thickness simulates the thickness of the vessel wall. Thetension on the synthetic tissue strip is adjusted to simulate thenatural anatomical vessel and its floppiness. The sutured strip can beimaged and the regularity of placement of sutures evaluated much moreaccurately than when using a tubular structure.

The apparatus may be constructed as a portable device that can be placedin the field of view of a surgical microscope or used with a surgicalloupe.

The apparatus may use two or more clamps to hold the tissue (latex,silastic tubes or natural body tissues). With clamps at two ends, thetissue is unsupported in the center where sutures need to be placed. Itmodels the floppiness of an unsupported tissue. By having two clampsspaced apart for clamping opposed ends of the tissue, it simulates thesituation of anastomosis where the tissue is held between clamps.

Placement of additional clamps can be added to simulate differentaspects of anastomoses. For example additional clamps and orienting thetissue in a T fashion simulates end to side anastomosis. The tissue willhave 4 ends in this case and the two pieces to be sutured together willform a ‘T’ shape.

In a further aspect the invention proposes a computer assistedalgorithmic evaluation of the suture which is objective and repeatable.

Existing imaging devices such as mobile phones and surgical microscopesmay be used as the image acquisition and transmission devices. To helpthe surgeon, a mobile application is used to guide the imaging process.Mobile application ensures clear and focused images are sent to theanalysis system.

The system may be kept private, it may be confined to the trainingfacility or it may be kept in the global public cloud. The system can beextended as a teaching tool by providing video of the procedures usingcorrect techniques and video of procedures that result in defectivesutures. Defective sutures are identified using the algorithmicevaluation.

The apparatus is positioned with tissue samples in a desiredorientation, and suturing is performed. An image of the suture and theorientation of the device is sent to an analysis system using the imagecapture and transmission system. The received images are rated, eithermanually or automatically and based on the rating and complexity of theprocedure, the images are ranked. A surgeon who performs the specificsurgery is given the score correspond to the suture image uploaded bythe surgeon. Surgeons can enroll into one or more study groups whoperform specific types of practices.

A large number of samples from a large number of surgeons may becollected and analyzed. It provides enough data samples to quantify theerrors that training surgeons make into suturing vessels. By capturingthe video of the actual surgery and analyzing the hand movements or toolmovements, the errors may be correlated with a specific type of handmovement or tool movement, hand or tool position and orientation, a toolgrasping technique or a combination of these.

Further, the system can also provide a test platform for new suturingand knot tying procedures. By measuring the completion time, suturingquality and the learning time, the merits of the new surgical procedurecan be quantitatively specified. By comparing with actual training dataover a large population of training surgeons, the new procedure can beeffectively qualified as a replacement for another type of procedure.

In a first specific expression of the invention, there is provided anapparatus as claimed in claim 1. In a second specific expression of theinvention, there is provided a method as claimed in claim 10. In a thirdspecific expression of the invention, there is provided a suturetraining system as claimed in claim 15.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated and described herein withreference to the various drawings, in which like reference numbers areused to denote like system components/method steps, as appropriate, andin which:

FIG. 1 is a block diagram of a suture training system according to afirst embodiment of the invention.

FIGS. 2(a) to 2(d) are views of the microtrainer training apparatus inFIG. 1 oriented to have the tissue in the horizontal plane.

FIG. 3(a) is an alternate implementation of the microtrainer trainingapparatus in FIG. 1 with 3 degrees of freedom for the tissue holder.

FIG. 3(b) is another alternate implementation of the microtrainertraining apparatus in FIG. 1 with 4 degrees of freedom.

FIG. 3(c) is a perspective view of the arm in the microtrainer trainingapparatus of FIG. 1.

FIG. 4 is a sketch of using the suture training apparatus of FIG. 2.

FIG. 5-10 are sketches of using the apparatus of FIG. 2 in variousorientations.

FIG. 11 is a block diagram of the image capture and transmission systemin FIG. 1.

FIG. 12-17 are views of the apparatus of FIG. 2 in various presetorientations.

FIG. 18 is a flow chart of pre-processing an image of a suture.

FIG. 19 is a plan view of a suture on a synthetic tissue strip.

FIG. 20 is a block diagram of the image filtering and pre-processingsystem in FIG. 1 for manual and semi-automated and automated evaluationof suture images.

FIG. 21 is a flow chart of grading a suture.

FIG. 22 is a block diagram of the components of training recommendationengine in

FIG. 23 is a flow chart of training recommendation process.

DETAILED DESCRIPTION OF THE DISCLOSURE

Anastomosis is the process of establishing communication between twostreams, for example, blood vessels or leaf veins. Anastomosis may bedefined as the natural, surgical, traumatic, or pathological formationof an opening between two normally distinct spaces or organs. In thehuman body, natural anastomosis recombines vessels that have previouslybeen branching out. For example, arterial anastomosis establishes thecommunication between the arteries, or branches of arteries. Palmararches, plantar arch, circle of Willis, intestinal arcades, labialbranches of facial arteries are examples of natural arterialanastomoses. Venous anastomosis establishes the communication betweenthe veins or tributaries of veins. For example, the dorsal venous archesof the hand and foot. Arteriovenous anastomosis (shunt) establishes thecommunication between an artery and a vein.

Similar to natural anastomoses, surgeons synthetically establishcommunication between vessel parts that are disrupted due to trauma.Surgeons perform anastomosis (stitching two tubular three dimensionalstructure) to permit vascular perfusion, nerve repair, during surgicalprocedures. For example, crossing anastomosis of a nerve is an effectivemethod to treat peripheral and central nerve injuries. Anastomosis isusually performed by micro surgeons using specialized fine instrumentson vascular and nerve structures of dimension 3 mm to 1 mm in diameter.The vascular and nerve structures are oriented in various planes in thebody. Surgeons need to orient the surgical tools and their hands invarious orientations to perform anastomosis. Surgical anastomosisconsists of a sequence of fine moves.

In order to place a stitch, micro or macro, the needle ideally entersthe tissue at 90 degrees, then the motion ideally follows the arc of theneedle, so that the needle exits the opposite edge of the tissue ideallyat 90 degrees to the surface. The path of entry and exit is ideallyperpendicular to the cut. This relationship between tissue surface andneedle ideally remains constant. When the orientation of the tissue ischanged, the posture of the hands has to change accordingly to bring theneedle and tissue in the correct relationship. This puts additionaldemands on the dexterity as the wrist, and fingers have a limited rangeof possible movements, i.e. forehand is the usual method for micro andbackhand is significantly difficult. For example forehand stitch on left45 degrees may be more difficult than placing a forehand in right 90degree orientation.

In order to provide suture training, a system 100 is shown in FIG. 1according to an embodiment. At each of three types of locations 110,120, and 130 a suture training apparatus 200 is provided. Location 110is a personal training location where the image acquisition andtransmission device 400 transmits the data to the cloud 150 through thepublic WAN 140. In location 120 an imaging and image transfer device 400sends an image or video of the suture to an image analysis system 600.It may compare the suture image features with the suture image featuresavailable in the Training Recommendation Engine 800 hosted in the cloud150 through the WAN 140. In location 130, the training recommendationengine 800 is hosted in a private cloud 160. Location 120 is typical ofa small surgery training facility such as a teaching hospital or a smallsurgery teaching facility. Location 130 is a typical multi-locationdedicated surgery training facility which conducts regular surgerytraining courses. The image analysis system 600 may rate and/or rank thesuture. Automated, semi-automated and manual rating may be provided torate and/or rank the suture. The image analysis system 600 may alsotrack the progress of the training. The image analysis system 600 may belocated on a central server or provided in a cloud computing environmentconnected via a wired or wireless LAN, WAN or the internet. Trainingrecommendation engine 800 has full functionality of an image analysissystem 600 and additional video processing and ranking ability.

FIG. 2 shows an example of the suture training apparatus 200 accordingto an embodiment. A tissue holder 240 holds the simulated tissue. Asupporting arm 230 has a first end 234 in clamp joint 250 with thetissue holder 240; and has a second end 232 in base joint 220 with abase 210. There can be an angle 204 in the supporting arm 230. Relativemovement of the tissue holder 240 about the supporting arm 230 andrelative movement of the supporting arm 230 about the base 210 allowadjustment of an orientation of the tissue with up to 6 degrees offreedom.

The base 210 can be fixed to a stable structure or a base platform (notshown). The base joint 220 together with the clamp joint 250 providesthe necessary freedom of movement. Depending on the application, thebase joint can be made with restricted freedom. The base joint 220 maybe fixed as in 260 of FIG. 3(a) or provided with 1 degree of freedom asin 270 of FIG. 3(b). FIG. 3(a) provides an apparatus with 3DOF and FIG.3(b) provides an apparatus with 4DOF. Other arrangements such as a hingeon the base joint are also possible depending on the requirements of theapplication.

The tissue holder 240 comprises a frame 244 with two clamps 242. The twoclamps 242 are mounted parallel to each other on the frame 244 withenough space in between for suturing of the tissue sample. The synthetictissue may be a latex strip 300, latex tube, silastic tube or naturaltissue, such as chicken sciatic nerve, can also be used. The tissue isheld between the clamps 242. The tissue is severed and unsupported atthe center to simulate a severed or damaged anatomical vessel.

In another embodiment of the invention, the tissue holder 240 containsthree clamps supporting two pieces of tissue, one with both endssupported through the clips and the other supported at one end. The freeend is used for practicing end-to-side anastomosis. In yet anotherembodiment, the tissue holders may move independently.

The base joint 220 incorporates a socket 222 on the base 210 thatengages a ball 224 extending from the second end 232. A ball-and-socketjoint is thus formed for the base joint 220, allowing free positioningof the arm 230 in three degrees of freedom. The base joint 220 isconstructed with sufficient friction for secure relative position of thearm 230 with respect to the base 210, which can be measured by markings216. The base joint 220 maintains its position unsupported, unlesssubsequently adjusted.

The clamp joint 250 includes a socket 252 on the frame 244 engaging aball 254 on the first end 234. Again a ball-and-socket joint is formedfor the clamp joint 250 for positioning the tissue holder 240. Thisclamp joint 250 is also constructed with sufficient friction forsecuring the relative position of the tissue holder 240 with respect tothe arm 230 unless force is applied to the arm 230 or the tissue holder240 allowing for positioning of the two clamps 242 in any orientationwith three degrees of freedom. A series of pre-set orientations may beused to simulate various clinical patient postures and surgery sites.Other joints or mechanisms may be provided to at least provide 3 degreesof freedom or 6 degrees of freedom depending on the requirements.

Other arrangements are possible. For example, the device may have afixed base joint and a replaceable tissue holder and a fixed holderjoint. The replaced tissue holders are in preset orientations. Here thejoints may have zero degrees of freedom, but various obliqueorientations are provided through multiple tissue holder arrangements.

An example of the suturing process is shown in FIG. 4. A tissue strip300 of suitable synthetic material such as latex (representing thetissue to be stitched) is mounted on the clamps 242. The clamps 242 withthe mounted synthetic tissue strip 300 are positioned at the desiredheight and orientation by using the joints 220,250. The apparatus 200 ispositioned under a microscope or magnifying device. The synthetic tissuestrip 300 is severed 302 or otherwise damaged and the trainee 304attempts to suture 306 the cut edges as accurately as possible with atool 308. FIGS. 5 to 10 show the trainee 304 attempting to suture 306the tissue strip 300 in a range of preset planes, which may allow animprovement in skills of suturing in different orientations.

The suture training apparatus 200 may be made of any suitable material,such as a thermoplastic rendering molding process, or may be made withmaterials such as wood, that can be machined to the desired form. Thecolor of the apparatus is chosen to have good contrast under amicroscope. The suture training apparatus 200 is preferably inexpensiveand can be owned by each surgeon and surgery student. It eliminates theneed for live animals or biological organs or tissues for training.

FIG. 11 shows an example of the image acquisition and transmissiondevice 400 according to an embodiment. An Image acquisition module 410includes a camera or other imaging device. An Image focus module 420 hasthe ability to focus the camera to obtain an orthographic view of thesuture along with the suture holder. Image Quality Evaluation Module 430assesses image quality by checking a histogram for contrast featuresand/or the ability to extract lines. Extraction of features may be usedin devices with enough CPU power. For example, in mobile computingdevices such as smartphones, the facility may be present.

The image preview module 440 helps to view the image before sending itfor further processing and analysis. Images or poor quality may bediscarded. It may also allow the surgeon to repeat the experiment if itis seen that the procedure did not go well. The image storage module 450stores images captured by the image acquisition application for latertransmission. This is useful to provide a preview of the images as wellas to operate the image acquisition device without needing connectivityto the processing system or to the cloud.

Image transfer module 460 connects the transmission device 400 to theimage processing system 600. The Image processing system 600 may beresident in the Image acquisition module 410, or in a separate computingdevice either deployed across the cloud 140 or offered as a localservice. Adapting to the various deployment scenarios is achievedthrough the image transfer module 460.

Once the suturing is complete, a digital orthogonal image is acquiredwith the Image Acquisition Module 410. The Image Focus Module 420 mayinclude: an imaging application which can detect the correctly focusedsuture image; a set of markings to measure the length of the suture andits orientation; a marking that gives the angular orientation of thedevice; and/or an imaging application that communicates with theanalysis system to send focused images. The markings may be fiducialmarkings. The fiducial markings may include markings on the base and/orthe joints to determine the location of the holder, markings to assistwith locating or orienting of the holder, markings to measure the lengthof the suture, markings to measure the orientation of the suture,markings to determine if the imaging meets minimum quality, markings todetermine the identity of the apparatus owner and other suitablemarkings.

An example of markings on the joints to determine the location of theholder, are shown in FIG. 3(c). The base joint markings 226 can be usedto determine the yaw or pitch angle of the arm 230, or by geometry theheight of the holder 240. The holder joint marking 256 can be used todetermine the pitch or roll angle of the holder 240.

For example the fiducial markings may allow the user to orient thetissue in a range or preset planes as shown in FIGS. 12 to 17. In FIG.13 the yaw angle scale on the base 210 shows the frame 244 is at a leftangle of 45°. FIG. 14 shows the frame 244 is at a height from the base210 measured by the base joint markings 226. In FIG. 16 the yaw anglescale on the base 210 shows the frame 244 is at a left angle of 45°.FIG. 16 shows the frame 244 is at a height from the base 210 measured bythe base joint markings 226. FIG. 15 shows the frame 244 is at rollangle of 45° measured by the holder joint marking 256. Pitch angle canalso be incorporated if the full 6 degrees of freedom are required to betrained.

The image capture process may involve a dedicated camera affixed in someway to the base 210. Alternatively a handheld camera or mobile phone maybe used to take the images. The mobile phone may be a smart phone, forexample, storing an application or “app” that is, written according tothe phone platform or operating system. The application may include acamera interface which invokes and focuses the camera on the suture. Theapplication may include visual or audible cues to help position thecamera correctly for an effective image to be captured. Good focus maybe determined when multiple lines are able to be detected in the sutureimage. Good focus may also be able to be determined based on the degreeof blur in the image.

The Quality Evaluation Module 430 may be included in a software process1400 to acquire suture images as shown in FIG. 18. The quality may beobtained by computing the contrast transfer or based on the featuresdetected by the software. In some cases, both focus and quality may bedetermined by being able to read a bar-code added on the tissue holder.The process 1400 adjusts imaging parameters (step 1402), and acquiresthe image (step 1404). If the image is in focus (step 1406), the imagequality is calculated (step 1408), and above the quality threshold (step1410) then the image is stored (step 1412) and transmitted to thecentral image analysis system 600.

Another measure of quality is determined as follows. Let H(i) be thehistogram of the image at intensity i, meaning the count of image pixelshaving intensity i. An intensity i is a peak, if H(i)>H(i−k) andH(i)>H(i+k) for value of k to a small integer such as 5. K is known asthe width of the peak. Using these definitions, if the peaks in thehistogram are determined to be three, then it is a tri-modal histogram,and the underlying image is trimodal. The image may be a tri-modal imagewith the three histogram modes occurring at the suturing device imageportion, suturing tissue strip 300 and suture 306. All three of beingdistinct and can easily be contrasted. One quality estimate is based onthe check whether the image is trimodal.

Another method is to check the relative size of the peaks. If i, j and kare the three peaks in the descending order of heights, then the ratiosH(i)/H(j) and H(j)/H(k) are computed to evaluate the quality of theimage.

Another method is to compute the Hough Transformation of the image. AHough transformation considers the intensity edges in the image andrecords them. For example, a Hough transformation of the imageclassifies the image pixels to belong to a line by being proximal to theline and above a specified intensity threshold. A line is specified byusing the perpendicular distance of the line with the origin (r) and theangle Θ that the line of length r makes with the positive x-axis. Thereare three pairs of markings on either side of the clip assembly. Eachmarking has two line segments forming a cross pattern. Two lines of thecross are perpendicular. The pairs of cross patterns on either side ofthe clip are similarly placed. In the Hough transform, there will belines that are parallel to each other which will have the same angle Θ,but different values of r. Detecting these lines in the HoughTransformation of the image assures that the image is of good quality.

FIG. 20 shows an example of the image filtering and pre-processingsystem 600 according to an embodiment.

The Image quality assessment module 430 may be present in the imageacquisition device 600 as well as on the processing device 400. Somehigh end devices may be capable of high quality processing, otherdevices may be limited. A full version of quality assessment maytherefore be implemented in a central location. Image quality assessmentis important for more accurate results in determining the quality of thesuture.

The purposes of the image filtering and pre-processing system 600 mayinclude assessing whether the image is of sufficient good quality and ifit meets the quality requirements, enhancing the image for betterprocessing and extracting features, such as the type of suture and theattributes of suturing, from the image.

To extract the image features, the region of interest, (the region,including the synthetic tissue and tissue holder) are extracted from theimage by segmenting the image. A set of landmarks is identified; forexample, the metal clips 248 of the tissue holder may act as a referencepoint. There are two segmentation modules, the Image Area SegmentationModule 625 which detects the area of an image which contains thesynthetic tissue, fiduciary markings 246 and the tissue holder and theImage Tissue Segmentation Module 610 which extracts the minimumrectangle containing the synthetic tissue and the suture. Instead ofusing a single segmentation, the use of multiple segmentation moduleswith different heuristics may increase accuracy in a wider range ofscenarios.

The Image Aspect Ratio Capturing Module 615 is used to present theregion of interest in a non-distorting magnification by enforcing acommon aspect ratio for the region of interest. The Image SizeCalibration Module 635 is used for converting the image dimensions inpixels, in the actual physical object dimensions in millimeters orfractions of millimeters.

Each device has a coded marking that indicates the device ID. Device IDis one of the targets of the segmentation. The Device IdentityRecognition Module 620 uses the coded markings to identify and decodethe device ID.

In a segmented image, special features are expected to be present. Forexample, while lines may be present in other parts of the image, thelines present in the synthetic tissue are highly likely to be thesutures. The segmentation operation, thus reduces the detection errorsthat could be caused by the automated image processing. The SutureOffset Identification Module 645, identifies the offset 340 of thesutured tissue ends. There may be one or two offsets based on the typeof suturing defect. Image suture extraction Module 630 extracts theimage coordinates of the suture knots 320 from the suture image. If thesuture image cannot be extracted, it Module 630 will extract the areacontaining the suture and present it for manual extraction of sutureknots by the human user, albeit marking the suture knot coordinates.Suture Segment Identification Module 650 combines topographicallyadjacent suture knots 320 to form suture segment 310. Each suturesegment is measured for length. Suture Contour Generator 660 determinesa line of best fit through the suture knots and can be used forassessing regularity.

The Affine Transformation 640 is a utility module used by other modulesto transform the image.

In general terms there are two main filtering preprocessing operations,(i) extract the image features and (ii) derive the suture quality andrank to yield a recommendation. Suture features may be extracted byidentifying the segment and offset lengths and converting these lengthsinto physical coordinates. For the conversion of the pixel (image)coordinates to physical coordinates, reference objects such as themetallic clip on the holder, clip holder width, fiduciary markings maybe used.

The suturing practice may also be video recorded to capture the surgicalmotion. The video is analyzed frame by frame to detect motion andproduce the movement. The recording should be done through the help ofthe microscope used in the practice.

The grading process may be carried out automatically, semi-automaticallyor manually with a Suture Image analysis System. The system may include:an analysis system that can analyze images and detect suture defects andextract procedure complexity level; a rating system that grades thesuture based on the defects or lack of thereof; and/or a ranking systemthat can rank the images across a collection of rated images and theprocedure complexity.

The image is stored by the central image analysis system 600 withsoftware installed to process each image automatically,semi-automatically or manually. First the entry and exit points of thesutures are identified, as well as the tissue edge junction whilevisually displaying the image. The user is then able to use the softwareto estimate inter-suture distance, angle of the suture and statisticssummarizing the quality of the suture. The results are displayed in atabular format on the screen and can be printed out.

The evaluated suture images are available only to the registered users.In some cases, a registered user will have a device ID. In someconfigurations, the device ID is bar coded on the tissue holder and isread using the image. A user is required to present the matching deviceID when accessing the suture images. A device ID may be shared bymultiple users. In other configurations, a registered user will have anorganization's identity and multiple training devices are registeredwith the organization.

The system also has a software program 1600 that visualizes the digitalimage of the synthetic tissue strip and allows processing on the imageto automatically or semi-automatically detect suture entry and exitpoints as well as the tissue edge junction as illustrated in FIG. 21.

The suture is evaluated using a single image of the suture. The image ofthe suture is acquired and transmitted for grading the suture 1602. Thegrading system first processes the images to improve the visual clarityto be examined by the automated process or through a manual process1604. In the next step, it crops the image to reduce the size of theimage 1606. The cropped image will have the suture and the clip assemblyin the image. The cropped image is stored for processing by theautomated software agent or by the manual grading or a combination ofboth.

Alternatively the suture may be evaluated using multiple images orvideos of the suture. Images may be taken from different viewpoints toextract a 3D or a pseudo 3D representation of the suture. For example animage may be taken from the front side of the suture and the back sideof the suture. This may be useful for more accurately measuring certainfeatures e.g.: evertion.

In the automated process, the suture is evaluated to derive the suturedistance, the distance of the suture entry and exit points in relationto the wound, the appearance of the wound surface 1608. The rated suturemay be ranked 1610 according to stored data or queued for manual grading1612.

In automated grading, the lines are extracted using one or more of thefollowing methods. Lines may be extracted using the filtering methods,for example, using a Sobel operator or Canary Filter. A filtered imagehaving lines of, for example, suture, holder assembly, tissue edges, aresubjected to line evaluator.

One approach to do line evaluation is using the Hough Transform totransform the image into a histogram of lines of varying r and Θ. Fromthe r values, the lines corresponding to suture images are heuristicallypicked. For example, the suture image is situated between the lines ofthe holder assembly. It is also placed between the cross markers on theholder assembly. Hence the r values must be between the linescorresponding to the right and left holder assemblies. The Θ values canbe very varying depending on the way the suture lines run.

Once the suture area is located using the line, histogram given by theHough Transform, a small area of the image corresponding to the sutureis subjected to the processing. The suture 306 is composed of eightsuture segments 310 as shown in FIG. 19. An ideal suture will have eachsegment is about the same length. For example, by placing a 4 mm widelatex specimen across the tissue holders, divided and sutured, thereshould be 9 suture knots 320, each marking an end of a suture segment310.

Good suturing involves regular 0.5 mm spacing (clamp width is 4 mm, a 4mm strip is recommended for use), and the trainee 304 should be able toplace 7 to 8 evenly spaced sutures 306. Suture bite should besymmetrical on both sides of the cut edge and the edges should beeverted not inverted. The degree of eversion/inversion, the pitch andits regularity forms the basis of the quality index. For example, in onemethod, the degree of eversion is calculated as the height inferred bythe shading of the image. In other cases, the width of the suture lineis treated as the degree of eversion. By calculating the distancebetween the entry points and exit points, the regularity of the entryand exit points are calculated. For example, the entry points may be ata pixel distance of 20 pixels apart on average and the maximum deviationof the entry point distance may be 2 pixels and is treated as havinggood regularity compared with one having a maximum deviation in theentry point distance of 5 pixels.

A good suture should be regular as well as be everted. One method tocheck whether the suture 306 is regular is to locate the suture knots320 and grade the suture segments 310. Based on the types of errors thatmay be made, a suture 306 is considered as comprising eight suturesegments 310 and two offsets 340 at either end of the suture segments310. If offsets 340 are present, then the suture 306 is not donecorrectly. If each of the suture segments 310 are of a specified length(4 mm into 8 segments gives each segment to be of 0.5 mm in size whichmay be graded for variation in segment length from 0.5 mm) then thesuture is done perfectly.

In the manual grading process, the image is moved to locate the suturearea into focus. Then a set of reference points is selected to createthe measurement units corresponding to the image pixels.

In one method, the distance between the ends of the clip holder istreated as a reference. The width of the clip holder is about 5 mm andaccordingly, the pixel dimensions may be made. In another approach, thedistance between the cross patterns on either side of the clip holderassembly may be taken. There are multiple distances that could be usedas reference.

Once the reference points are marked, region of interest is marked toinclude the suture area. The image is then magnified with the suturearea alone. Then the offset if any is marked. An offset arise if thereis a mismatch between the two ends of the tissue being sutured. At mosttwo offsets can occur. Then the suture knots are marked to show thesuture segments. The system calculates the segment distance.

At the end of the grading, a colored ribbon with 10 segments isproduced, two segments for the offset errors and 8 segments for thesuture segments.

In Semi-automatic Grading mode, the system helps to identify the regionof interest and points of interest such as the suture knots, holderassembly ends, suture markings. The manual grader may augment theseidentified points and extract the 8 sutured segments and offset errors.The offset may be marked as the difference between two tissue ends inthe suture. In some sutures there will be no measurable offset on eitherend of the suture and in some there is offset in one end of the sutureand in others there are offsets on both ends of the sutures. In bothmanual and semi-automatic modes, the grader may award bonus points forevertion.

A Scoring Algorithm may be applied to the image to provide a score ofthe suture of the image. Using one of the methods (manual, automatic orsemi-automatic) methods, the suture is extracted into a graded ribbonrepresentation. There are typically 10 segments in the ribbon, 8segments for the suture and 2 segments for the offset. The scoringalgorithm examines the ribbon and assigns scores for each segment.

In one method of the scoring algorithm, the scores are computed asfollows, where O1, O2, S1, S2, . . . , S8 represent the ribbon segments.Score(Si)=max(10−|(length(Si)−0.5 mm)|*2.0,0)Score(Oi)=10 if length(Oi)==0[0100]Else min(0,−length(Oi)*2.5)Total Score=sum(Score(Si)) for I=1 . . . 8+sum(Score(Oi)) for I=1 . . .2

Another implementation of the algorithm considers the standard deviationacross length of segments. For example, if the segments are {0.6, 0.5,0.4, 0.6, 0.7, 0.4, 0.5, 0.3}, then the standard deviation of thesegment lengths is calculated by calculating the mean and finding thesum of squared deviations from the mean. Based on this, each segment maybe given a score based on the closeness to the standard deviation. Forthe above example, the mean is 0.5 and standard deviation is 0.130931.Hence segments with length 0.3 and 0.7 will be punished in the scoring.

In yet another implementation, the value of the standard deviationitself is taken as a score.

The scoring scheme may also be augmented using minimum number ofsegments needed to consider a valid suture. For example, a suture withless than 4 segments is scored as zero.

FIG. 22 shows an example of the Training Recommendation Engine 800according to an embodiment. The Training Recommendation Engine 800 mayinclude a reporting system that keeps the progress of a surgeon overtime and indicate the progress; a grouping system that can match andpair surgeons with complementary skills to develop the skills of both;and/or a video indexing system that can index suture defects to videosexplaining suturing defects and remedy. After obtaining the suturequality in terms of the segment lengths and offsets, it may becorrelated to suturing techniques. Techniques that give better precisionare recommended. A ribbon representation of the suture is produced foreasier visual comparison between training sessions.

The Ribbon Generator 805 may generate set of lengths from a sequence ofpixel coordinates indicating suturing knots, and apply the scoringalgorithm to create a color coded strip. The ribbon generator 805provides input to a Suture View Generation System 825 to provide arectangular image corresponding to the suture with the number ofrectangles equal to the number of segments in the suture.

The Pitch Generator 810 determines the regularity with which the suturesare made. The Pitch may be the mode (most common) of the segmentlengths. Each segment length may be matched to one of four segments 310sizes and the most common segment size is used as the output of thepitch generator 810.

A Depth Generator 815 processes an area of the image and determines themaximum depth of the area by comparing the pixel intensity variationacross the area, while maintaining continuity between the adjoiningregions.

Suture Grading and Ranking Module 820 grades the suture based on therectangular image, pitch and depth. Rated suture images are comparedwith other suture images to produce a ranking order. A composite scoremay be assigned to the images based on the output of the Ribbon, Pitch,Depth generators 805,810,815 and the images are sorted according to thenon-increasing value of the score.

The Suture View Generation System 825 produces a suture view bysuperimposing extracted features on the extracted suture image. Thesuperimposed features may include the regularity of the suture segments,the composite score and rank of the suture, a visual ribbonrepresentation of the suture and markings to show suture knots.

The generated view of the image is cached or stored for each givensuture using two storage facilities in the Training RecommendationEngine 800. The Image/Video Storage 830 is for image and video pixeldata storage and the Image/Video Feature Storage 835 is for theextracted data. The Image/Video Storage 830 contains the original imageas well as images obtained using various segmentation modules. TheImage/Video Feature Storage 835 contains data such as the ribbon data,the pitch data, the depth map, and other associated information such asthe time taken to perform the procedure, the training device used in theprocedure, the orientation of the device during the procedure.

The training Recommendation Generator 870 includes image preprocessing,feature extraction and user view generation. A recommendation of aprocedure is generated by the system based on the analysis of themovement. Surgical movement data is analyzed using three video analysiscomponents.

The Video processing and filtering system 840 processes individual videoframes for quality and assess the frame delay. It converts the videointo a sequence of images that can be analyzed using the imageprocessing and filtering system. The module focuses the video segmentson the microsurgical movements.

The Video Frame Analyzer 845 segments the frame into object landmarkswhich are assessed for movement in subsequent frames. For example, thelocation of the tip of the needle or the needle holder may be identifiedin each frame.

The Video Motion Detector 850 is used to detect movement of objectsidentified by the video frame analyzer 845. The movement is trackedacross each frame of the video data.

The Motion Data Generator 855 determines coordinate data of objectmovement across frames. This can be used to approximate the surgicalmotion.

The Motion View Generation System 860 produces a view of the surgicalmotion from the motion data generator 855 to a workspace. The surgicalmotion may be restricted in a small bounding cube of surgical movements.This view data may be used in rating or ranking to compare to othertraining data (or idealized motion data), it may be used in explainingmotion errors to the trainee or it may be used in identifying peers.

As shown in FIG. 23, the training recommendation engine 800 may executesoftware 1800 that approves suture parameters (step 1802), checks if thequality is above a threshold (step 1804), rates the image and rank (step1806), checks if a complimentary pair is available (step 1808), and ifnot, recommends training videos (step 1810). If the quality is below thethreshold (step 1804), the image is discarded (step 1812). If there is acomplimentary pair available (step 1808), a training pair is recommended(step 1814). The Suture View Generation System 825 allows viewing thesuture images with the ribbon representation of the suture. The ribbonrepresentation uses color codes and ribbon lengths to give a summaryfeedback to the practicing surgeon. In one implementation of the ribboncoding, a perfect suture has 8 green segments of equal size and twooffsets on either side with zero size. The equal sized segments may becolored green, for example. Other encoding schemes are possible.

A non-perfect ribbon will have different sized (lengthwise) segmentsthat are variously colored. Some ribbons may have fewer than 8 segmentsand some may have more than 8 segments. There may be non-zero sizedoffset segments as well. By examining the segments, a one-to-one mappingbetween the segments and the image is provided. For example, by hoveringover the segment, a corresponding portion of the suture is highlighted.Hence the surgeon will be able to analyze the errors in his suture usingthe recommendations provided in the ribbon.

The Training Recommendation Engine 800 may use the ribbon to findsurgeons who have complementary errors, and recommend peering forimproving the techniques of both the surgeons. By storing the trainingdata over the days, the progress of the surgeon is monitored; forexample: whether the errors are reduced over the training period. Twotypes of graphs are used to give feedback and recommendations to thesurgeon. The score of the surgeon over the period of the training andindividual points of the training. When individual points are examined,the surgeon can see the image with the ribbon annotation.

The surgeon is also provided by the segment length over time for each ofthe eight segments. The segment lengths of individual segments maycorrespond to a specific skill in suturing.

Whilst exemplary embodiments have been described in detail, manyvariations are possible within the scope of the invention as claimed aswill be clear to a skilled reader. East embodiment may be independentand not necessarily used with the other embodiments mentioned. Forexample a fixed inclination latex strip holder may be inserted into abase with a vertical slit. This holder may then be used with the imagingsystem and/or suture recommendation engine. In a further alternative,the rating system may compare the image against candidate good sutures.After aligning both the patterns to have the same starting point, thevariance can be determined to compute a rating.

Although the present disclosure has been illustrated and describedherein with reference to preferred embodiments and specific examplesthereof, it will be readily apparent to those of ordinary skill in theart that other embodiments and examples may perform similar functionsand/or achieve like results. All such equivalent embodiments andexamples are within the spirit and scope of the present disclosure, arecontemplated thereby, and are intended to be covered by the followingclaims.

What is claimed is:
 1. A system of microsuture training, the systemcomprising: a suture training apparatus configurable with simulatedtissue to simulate natural anatomical tissue encountered in actualsurgery; an image analysis system configured to receive a video of asuture performed using the suture training apparatus and the simulatedtissue; a feature extraction system configured to extract physicalfeatures of the suture from the video; a suture grading systemconfigured to score the suture based on the extracted physical featuresof the suture and based on a rating and complexity of the suture todetermine a quality of the suture in an objective manner; arecommendation engine configured to recommend training based on thequality and the extracted physical features; and a movement trackingsystem configured to track hand or tool movements from the video basedon a plurality of fiducial markings on the suture training apparatus,and based on the hand or tool movements, identify errors selected fromthe group consisting of: hand or tool movement, hand or tool positionand orientation, tool grasping technique, or a combination thereof,wherein each of the image analysis system, the feature extractionsystem, the suture grading system, the recommendation engine, and themovement tracking system is executed on one or more servers, and whereinthe plurality of fiducial markings include fiducial markings on a baseof the suture training apparatus providing an angle of a holder of thesuture training apparatus relative to the base and fiducial markings onjoints of the suture training apparatus providing (1) yaw or pitch of anarm of the suture training apparatus and (2) pitch or roll of theholder.
 2. The system of claim 1, further comprising a local database ofsuture images, suturing motion elements, and training recommendationscommunicatively coupled with a plurality of image analysis systems,wherein the suture grading system is configured to rank the suturesbased on a plurality of databases.
 3. The system of claim 1, wherein thefeature extraction system, the suture grading system, and therecommendation engine are implemented in a cloud.
 4. The system of claim1, wherein the physical features are extracted by generating a ribbonrepresentation for each of a plurality of segments of the suture, andwherein the quality of the suture is scored by comparing a length ofeach of the ribbon representations with a predetermined optimal lengthor by calculating a standard deviation of the lengths of the ribbonrepresentations.
 5. The system of claim 1, wherein the suture is scoredby determining a degree of eversion of tissue, determining a pitch ofthe sutures, determining a regularity of the sutures, determining alocation of knots, and determining if alignment errors are present. 6.The system of claim 1, wherein the physical features are extracted byidentifying a plurality of segments of the suture, measuring offsetlengths of edges of simulated tissue, converting the offset lengths intophysical coordinates, and determining image coordinates of suture knotsof the suture.
 7. The system of claim 1, wherein the quality of thesuture is based on spacing of the suture, symmetry of the suture, degreeof eversion/inversion, pitch, and offset of tissue.
 8. The system ofclaim 1, wherein the suture training apparatus is configurable with aplurality of orientations of the simulated tissue, wherein the pluralityof orientations are objective based on one or more fiducial markings onthe suture training apparatus.
 9. The system of claim 8, wherein therating and complexity of the suture are based in part on an orientationof the simulated tissue when the suture was performed.
 10. A method ofmicrosuture training, the method comprising the steps of: providing asuture training apparatus for use by a trainee, the suture trainingapparatus configurable with simulated tissue to simulate naturalanatomical tissue encountered in actual surgery; receiving a video of asuture performed using the suture training apparatus and the simulatedtissue; extracting physical features of the suture from the video;scoring the suture based on the extracted features of the suture andbased on a rating and complexity of the suture to determine a quality ofthe suture in an objective manner; recommending training based on thequality and the extracted physical features; and tracking hand or toolmovements from the video based on a plurality of fiducial markings onthe suture training apparatus, and based on the hand or tool movements,identifying errors selected from the group consisting of: hand or toolmovement, hand or tool position and orientation, tool graspingtechnique, or a combination thereof or identifying preferred movementsin surgery, wherein the plurality of fiducial markings include fiducialmarkings on a base of the suture training apparatus providing an angleof a holder of the suture training apparatus relative to the base andfiducial markings on joints of the suture training apparatus providing(1) yaw or pitch of an arm of the suture training apparatus and (2)pitch or roll of the holder.
 11. The method of claim 10, wherein thestep of extracting physical features comprises extracting the sutureinto a graded ribbon representation having a plurality of ribbonsegments, and wherein the step of scoring the suture comprises comparinga length of each ribbon segment with a predetermined optimal length orcalculating a standard deviation of the lengths of the ribbon segments.12. The method of claim 10, wherein the step of scoring the suturecomprises determining a degree of eversion of tissue, determining apitch of the sutures, determining a regularity of the sutures,determining a location of knots, and determining if alignment errors arepresent.
 13. The method of claim 10, wherein the step of extractingphysical features comprises: identifying a plurality of segments of thesuture; measuring offset lengths of simulated tissue; converting theoffset lengths into physical coordinates; and extracting imagecoordinates of suture knots of the suture, entry and exits points, andsuture line of the suture, wherein the image coordinates are used toscore the suture.
 14. The method of claim 10, wherein the quality of thesuture is based on spacing of the suture, symmetry of the suture, degreeof eversion/inversion, pitch, and offset of tissue.
 15. The method ofclaim 10, wherein the step of scoring the suture comprises displaying animage of the suture to a user for manual grading.
 16. The method ofclaim 10, wherein the suture training apparatus is configurable with aplurality of orientations of the simulated tissue, wherein the pluralityof orientations are objective based on one or more of the pluralityfiducial markings on the suture training apparatus.
 17. The method ofclaim 16, wherein the rating and complexity of the suture are based inpart on an orientation of the simulated tissue when the suture wasperformed.
 18. The method of claim 10, wherein the one or more serversreside in a cloud.
 19. A suture training system, comprising: a suturetraining apparatus configurable with a plurality of orientations ofsimulated tissue to simulate natural anatomical orientations encounteredin actual surgery, wherein the plurality of simulated tissueorientations are objective based on one or more of a plurality offiducial markings on the suture training apparatus; an image analysissystem configured to receive a video of a suture performed using thesuture training apparatus and the simulated tissue; a feature extractionsystem configured to extract physical features of the suture from thevideo; a suture grading system configured to score the suture based onthe extracted physical features of the suture and based on a rating andcomplexity of the suture to determine a quality of the suture in anobjective manner, the rating and complexity of the suture based in parton the orientation of the simulated tissue when the suture wasperformed; and a recommendation engine configured to recommend trainingbased on the score and the extracted features, wherein each of the imageanalysis system, the feature extraction system, the suture gradingsystem, and the recommendation engine are executed on one or moreservers, and wherein the plurality of fiducial markings include fiducialmarkings on a base of the suture training apparatus providing an angleof a holder of the suture training apparatus relative to the base andfiducial markings on joints of the suture training apparatus providing(1) yaw or pitch of an arm of the suture training apparatus and (2)pitch or roll of the holder.
 20. The suture training system of claim 19,wherein the feature extraction system, the suture grading system, andthe recommendation engine are implemented in a cloud.